-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor: Update DataframeAbstract
#547
Refactor: Update DataframeAbstract
#547
Conversation
* (refactor): update behaviour of `to_dict()` method, make it possible to accept `into` and `as_series` parameters (the last one is for polars dataframes). * (tests): add tests for casting the dataframe to a dictionary, add tests for passing parameters in the proxy-call to dataframe's `to_dict()` method.
* (docs): add docstrings for proxy-methods in `DataframeAbstract` class
Codecov Report
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. @@ Coverage Diff @@
## feature/v1.2 #547 +/- ##
================================================
+ Coverage 83.39% 83.42% +0.03%
================================================
Files 52 52
Lines 2584 2589 +5
================================================
+ Hits 2155 2160 +5
Misses 429 429
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
* (chore): update return type for `df_type` function
WalkthroughThis pull request enhances the flexibility and readability of the pandasai package. It modifies the return type of Changes
TipsChat with CodeRabbit Bot (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Files selected for processing (3)
- pandasai/helpers/df_info.py (1 hunks)
- pandasai/smart_dataframe/abstract_df.py (3 hunks)
- tests/test_smartdataframe.py (3 hunks)
Files skipped from review due to trivial changes (2)
- pandasai/smart_dataframe/abstract_df.py
- tests/test_smartdataframe.py
Additional comments (Suppressed): 1
pandasai/helpers/df_info.py (1)
- 14-14: The function signature has been changed from returning
str
toUnion[str, None]
. Ensure that all calls to this function throughout the codebase can handle aNone
return value.
Thanks a lot @nautics889 , great improvement (in particular the |
* feat: add SQL connectors * feat: add .env configuration for mysql and postgres * feat: a smart df can now accept a connector as a param * fix: correct error prompt now handles multiple dataframes * feat: cache the dataframes from third parties * fix: truncate long text from the sample * feat: extracting filters (#451) (#483) * fix: setters were not overriding some properties in some cases * fix: remove multiple df overwrites (#436) * improved on _is_df_overwrite method to check all targets * test: add test for multiple df overwrite --------- Co-authored-by: Gabriele Venturi <[email protected]> * fix: prevent to_* pandas methods from running * Release v0.8.3 * fix: environment for executing code (#419) * fix: environment for executing code * (fix): set `matplot.pyplot` package under "plt" alias in `_get_environment()` * feat: Try to import a package when NameError (#430) * (refactor): split calling of `exec()` for code and handling of errors into separate methods; * (feat): add trying to import the `exc.name` and add one to context when `exc` is an instance of `NameError`; * (tests): add tests for new methods; * fix: add logging for `NameError` handling (#430) * (fix): add log a general exception raised when trying to fix `NameError` * fix: compatibility with python 3.9 (#430) * (fix): update `handle_error()` method, add parsing `exc.args[0]` for cases with NameError if there is no attribute "name" of object of the exception. * fix: update for `handle_error()`(#430) * (fix): add checking if the library to be imported is preset in the whitelist of third-packages; * (fix): remove "plt" when forming environment for code to be run; * (fix): add raising an exception that was caught during fixing code with the error correction framework; * (tests): add test-case for retrying to fix code with the error correction framework unsuccessfully; * (tests): fix issue with inappropriate instantiation of numpy's `array` object within several test methods; * Release v0.8.4 * Release v1.0b1 * Release v1.0 * fix: convert polars to pandas df and back to polars after exec the code * docs: update docs for v1 * Release v1.0.1 * fix: remove shortcuts from backwords compatibile version * Release v1.0.2 * refactor: minor updates (#446) * (refactor): inappropriate type hint for `_has_run` in base class for middlewares; * (style): remove built-ins name shadowing in `__init__()` method of `Prompt` class; * docs: update LLMs documentation code to match the last version (#445) I have updated OpenAI, Starcoder, Falcon, and AzureOpenAI. But I haven't edited GooglePalm because it still uses api_key in the last version not api_token Co-authored-by: Gabriele Venturi <[email protected]> * fix: do not override setters and getters with __getattr__ * Release v1.0.3 * fix: huggingface models (#454) * Release v1.0.4 * fix: update code manager to convert prompt id to string (#462) * test: increase test coverage (#460) * Update LLMs documentation code to match the last version I have updated OpenAI, Starcoder, Falcon, and AzureOpenAI. But I haven't edited GooglePalm because it still uses api_key in the last version not api_token * Create test_google_vertexai.py Create test cases for Class Google Vertexai. I have tested initailization function with default model provided and custom one as well. Then I have built test cases for validation of class when model provided and without model. * Update test_smartdataframe.py Created test cases for _load_df() and _import_from_file() functions * test: remove verbose test * test: unskip skipped test * test: test multiple invalid file formats * test: fix tests --------- Co-authored-by: Gabriele Venturi <[email protected]> * fix: recursive error on df last_result property (#468) (#469) * fix: recursive error on df last_result property (#468) * fix: change last_result type to str * test: add tests --------- Co-authored-by: Gabriele Venturi <[email protected]> * Release v1.0.5 * fix(llms): AzureOpenAI call method signature (#476) * fix: added last_error property reference in smart_datalake (#475) * fix: temporarely disable multi-turn capacity, causing hallucinations * Release v1.0.6 * fix: create required folders as the datalake is initialized * Release v1.0.7 * fix: create required folders in the project folder * Release v1.0.8 * chore: fix typing in the logger logs (#464) * docs: add video explanation for SmartDataframe * feat: extracting filters (#451) * (feat): add `_extract_filters()`, `_extract_comparisons()`, `_tokenize_operand()` methods to `CodeManager`; * (tests): add basic tests; * test: extracting filters (#451) * (tests): rename `test_extract_filters()` to `test_extract_filters_col_index()`, update asserting according to task requirements; * (tests): add `test_extract_filters_col_index_multiple_df()` for multiple dfs; * fix: update according to requirements (#451) * (feat): add module `node_visitor.py`, add `AssignmentVisitor` in `node_visitors.py`; * (fix): `_extract_filters()` now returns dictionary; * (feat): add grouping by df number when forming; * (tests): update naming in test cases; * (tests): add test method with non default (not `df`) variable name; * feat: extracting filters (#451) * (feat): add `CallVisitor` to `node_visitors.py` module; * (feat): add extracting comparisions in `filter()` method being called as an attribute of object named `pl` or `polars`; * (tests): test methods for extracting polars-like filters; * refactor: extracting filters (#451) * (refactor): add handling exceptions in `_extract_filters()` method; * docs: extracting filters (#451) * (docs): add documentations for implemented functions; * (style): type hinting; * test: merge similar tests --------- Co-authored-by: Gabriele Venturi <[email protected]> Co-authored-by: Aymane Hachcham <[email protected]> Co-authored-by: Omar Elsherif <[email protected]> Co-authored-by: Hanchung Lee <[email protected]> Co-authored-by: Sanchit Bhavsar <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]> * docs: add example of usage with connectors * feat: allow to save and load config for connectors * feat: add dynamic filters to connectors * feat: add yahoo finance connector * fix: prevent sql injections * refactor: improve sql connector * test: test yahoo finance connector * refactor: update `DataframeAbstract` (#547) * refactor: `to_dict()` parameters * (refactor): update behaviour of `to_dict()` method, make it possible to accept `into` and `as_series` parameters (the last one is for polars dataframes). * (tests): add tests for casting the dataframe to a dictionary, add tests for passing parameters in the proxy-call to dataframe's `to_dict()` method. * docs: docstrings for `DataframeAbstract` * (docs): add docstrings for proxy-methods in `DataframeAbstract` class * chore: type hint issue * (chore): update return type for `df_type` function * feat: improve prompt in the way it handles conversation and dataframes --------- Co-authored-by: Ihor <[email protected]> Co-authored-by: Aymane Hachcham <[email protected]> Co-authored-by: Omar Elsherif <[email protected]> Co-authored-by: Hanchung Lee <[email protected]> Co-authored-by: Sanchit Bhavsar <[email protected]> Co-authored-by: Massimiliano Pronesti <[email protected]>
to_dict
method inDataframeAbstract
. Addinto
parameter (defaults todict
) since that parameter is present in originalpandas.core.frame.DataFrame.to_dict
. Addas_series
parameter (defaults toTrue
) for polarsDataFrame
. Add engine type check: depending onself._engine
type, the call is going to be proxied either topandas.core.frame.DataFrame.to_dict
orpolars.DataFrame.to_dict()
.to_dict()
. Add a test for checking if the type casting to dictionary works. Add a test for checking if the parameters are passed well according to the._engine
type ofSmartDataframe
.DataframeAbstract
class.Summary by CodeRabbit
df_type
function inpandasai/helpers/df_info.py
to returnNone
for invalid or None dataframes, enhancing error handling._engine
inDataframeAbstract
class inpandasai/smart_dataframe/abstract_df.py
to handle different dataframe engines.DataframeAbstract
class improving code readability and understanding.tests/test_smartdataframe.py
for testing theto_dict
method ofSmartDataframe
with various parameters.